An Empirical Evaluation of Bagging and Boosting

An ensemble consists of a set of independently trained classi ers such as neural networks or decision trees whose predictions are combined when classifying novel instances Previous re search has shown that an ensemble as a whole is often more accurate than any of the single classi ers in the ensemble Bagging Breiman a and Boosting Freund Schapire are two relatively new but popular methods for produc ing ensembles In this paper we evaluate these methods using both neural networks and decision trees as our classi cation algorithms Our results clearly show two important facts The rst is that even though Bagging almost always produces a better classi er than any of its individual com ponent classi ers and is relatively impervious to over tting it does not generalize any better than a baseline neural network ensemble method The second is that Boosting is a powerful technique that can usually produce better ensembles than Bagging however it is more susceptible to noise and can quickly over t a data set